Node-based Induction of Tree-Substitution Grammars

Traditionally, syntactic parsing is done using probabilistic context-free grammars (PCFGs) or variants thereof, as there are standard efficient methods for parsing with PCFGs and for extracting them from a corpus. However, PCFGs do not accurately represent many dependencies in natural language. For example, many determiners can only occur with certain types of nouns. Determiners like a and another can only occur with singular count nouns, while those can only precede plural nouns, and determiners like more can precede either plural nouns or mass nouns but not singular count nouns. To represent these dependencies with a PCFG, we must have many separate categories for both determiners and nouns, as a simple rule like NP → DT N (where DT stands for determiner and N stands for noun) will overgenerate noun phrases like a water or those cat. One formalism that makes representing long-range dependencies much simpler is probabilistic tree-substitution grammars (PTSGs). While every CFG rule can be seen as one level of a syntactic parse tree, and thus a subtree of height 2, TSG rules can be any subtree of a syntactic tree, thus allowing them to concisely represent dependencies that involve more than one level of syntactic structure. For example, a TSG for generating noun phrases can represent what types of nouns common determiners can precede using rules like the following: NP